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INTRODUCTION 

Developers of attitudinal questionnaires/ scales (of which questionnaires that 
compute satisfaction with usability are one type) are trained to consider 
questionnaire response styles such as extreme response bias and acquiescence 
bias. In acquiescence bias, respondents tend to agree with all or almost all 
statements in a questionnaire (Lewis and Sauro, 2009). The acute response bias is 
the inclination to mark the extremes of rating scales rather than points near the 
middle of the scale. To the amount that these biases exist, the affected responses 
do not provide a true measure of an attitude. Acquiescence bias is of particular 
apprehension because it leads to an upward error in measurement, giving 
researchers too positive a picture of whatever attitude they are measuring. 

An approach commonly employed to lessen the acquiescent response bias is the 
inclusion of negatively worded items in a questionnaire/ scale. Questionnaires 
with a mix of positive and negatively worded statements force attentive 
respondents to disagree with some statements. Underneath the assumption that 
negative and positive items are essentially equivalent and by reverse scoring the 
negative items, the resulting composite score should have reduced acquiescence 
bias. More recently, however, there is evidence that the strategy of including a 
mix of positively and negatively worded items creates more problems than it 
solves. Such problems include lowering the internal reliability, distorting the 
factor structure/ construction and problems of criterion related validity (Lewis and 
Sauro, 2009). 

The validity and reliability are the basic concepts in the educational measurement, 
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because the validity and reliability represent the most important elements and 
conditions that must be measured in measurement tools to be used. It relates to the 
interpretation of scores from psychometric instruments (e.g., symptom scales, 
questionnaires, education tests, and observer ratings) used in research. 

Criterion-related validity covers correlations of the tool with another criterion 
too I. which is accepted as valid (A’llam, 2006). In other words, criterion-related 
validity is where a correlation exists between the scores on a tool and the scores 
on other existing tools which are accepted as valid. (Al-thebaity, 1998). A test is 
valid when it does not test anything else rather than the purpose it has been 
designed for. When a test is designed to test the mathematical skill of a 
pupil/student, it should really test that skill and does not test or evaluate other 
additional skills like linguistic competency or mathematical speed. Otherwise the 
test would be considered as invalid, (Abu-zeinah, 1998). 

The concept of construct validity was introduced by Cronbach and Meehl (1955). 
A construct is built in the minds of psychologists and does not exist in any 
concrete way, but it does exist in a theoretical way as an idea. Constructs are 
things like intelligence, anxiety, depression or thinking. In establishing concurrent 
validity, scores on an instrument correlate with scores on another (criterion) 
measure of the same constmct or a highly related construct that is measured 
concurrently in the same subjects. (Kimberlin and Winterstein, 2008) 


RESEARCH OBJECTIVES 

1. To examine the effect of (independent variable) items direction (positive or 
negative) on the factorial construction related validity (dependent variable) in 
Likert scale. 

2. To examine the effect of (independent variable) items direction (positive or 
negative) on the criterion related validity (dependent variable) in Likert scale. 


RESEARCH HYPOTHESES 

1. There is no effect of items direction (positive or negative) on the factorial 
construction related validity in Likert scale. 

2. There is no effect of items direction (positive or negative) on the criterion 
related Validity in Likert scale. 
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RESEARCH METHOD 

To achieve the above mentioned objectivities of study, the researcher used 
descriptive survey method. As it is common approach used in the field of social 
and human sciences. Descriptive research observes and records carefully a certain 
phenomenon or problem during certain periods of time with the purpose of 
exploring such problem in terms of content and characteristics to reach certain 
conclusions and generalizations which can help in understanding the current 
situation and improving it, (Alian, 2001, P: 47). 


Research Sample 

The random sampling technique was used for selecting (510) undergraduate 
students from the College of Education at King Saudi University, Riyadh, Saudi 
Arabia, in the academic year (2012- 2013). 


Research Tool Used 

An attitude scale developed by (SobheiHamdan Abu Jalalah& Mohammed 
JehadGamel, 2007) was used to know the attitude of students towards 
Undergraduate study and another scale of academic adjustment prepared by them 
has been used as criterion. 

a- Ensuring validity of the tool. 

There are many definitions for validity, (Pallant, 2011) defines the validity as “the 
degree to which it measures what it is supposed to measure. Unfortunately, there 
is no one clear-cut indicator of a scale’s validity”. To ensure and test the validity 
of the tool the researcher used the criterion related validity. 

Criterion-related validity covers correlations of the tool with another criterion 
too I. which is accepted as valid (A’llam, 2006). In other words, criterion-related 
validity refers to the accuracy of, a measure or procedure by comparing it with 
another measure or procedure that has been demonstrated to be valid. ( Pierangelo 
and Giuliani, 2008) 

The validity of this tool has been evaluated by using criterion related validity and 
for that Pearson correlation coefficient was used. The validity coefficient was 
(0.61) 
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b- Ensuring reliability of the tool 

Cronbach (2004) believes that reliability as the correlation of an instrument with 
itself. To evaluate the reliability of research tool Alpha Cronbachwas used. It was 
calculated as (0.82). 


Research process 

Moreover, the researcher prepared different copies of the research tool (an attitude 
scale to know students attitude towards Undergraduate study) according to the 
items’ direction dissimilarity (positive, negative, half- positive, half- negative and 
random [positive & negative]) 


Statistical methods 

- Alpha Cronbach Coefficient 

- Factor Analysis 

- Fisher Equation, to estimate the differences between the validity coefficients 


RESULTS AND INTERPRETATION 
1. First Objective 

To exa min e the effect of items direction (positive or negative) on the 
factorial construction related validity in Likert scale. 

To answer the objective above, the researcher used factor analysis for each form 
(positive, negative, half- positive, half- negative and random [positive & 
negative]) and the result was as following: 

Table 1 Factorial Construction 


Items Direction 

Number of 

Factors 

Cumulative % 

Positive 

9 

74.89 

Negative 

8 

76.32 

Half positive and Half- 

negative 

9 

73.99 
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Random [positive & negative] 


8 


70.94 


Table no. {1} showed that there are differences in the number of factors 
calculated by factor analysis andthe null hypothesis, “there is no effect of items 
direction (positive or negative) on the factorial construction related validity in 
Likert scale” was rejected. 


Table 2 Positive Form Factor Analysis 


Component 

Total 

% of Variance 

Cumulative % 

1 

6.341 

23.483 

23.48 

2 

3.279 

12.143 

35.63 

3 

2.398 

8.883 

44.51 

4 

2.089 

7.737 

52.27 

5 

1.518 

5.623 

57.87 

6 

1.322 

4.898 

62.77 

7 

1.166 

4.319 

67.09 

8 

1.077 

3.990 

71.08 

9 

1.029 

3.811 

74.89 


The table 2 above shows that there are 9 factors of positive form of items and they 
contributed 74.88% of cumulative. 

Table 3 Negative Form Factor Analysis 


Component 

Total 

% of Variance 

Cumulative % 

1 

8.514 

31.534 

31.534 

2 

2.638 

9.771 

41.305 

3 

2.211 

8.189 

49.493 

4 

1.817 

6.731 

56.224 

5 

1.678 

6.214 

62.438 

6 

1.462 

5.414 

67.853 

7 

1.185 

4.388 

72.241 

8 

1.101 

4.079 

76.320 


The table 3 above shows that there are 8 factors of negative form of items and 
they contributed 76.32% of cumulative. 


Table 4 Half Positive and Half Negative Form Factor Analysis 


Component 

Total 

% of Variance 

Cumulative % 

1 

4.993 

18.492 

18.492 
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2 

3.189 

11.811 

30.303 

3 

2.474 

9.163 

39.466 

4 

2.248 

8.325 

47.791 

5 

1.946 

7.206 

54.997 

6 

1.542 

5.711 

60.707 

7 

1.338 

4.957 

65.665 

8 

1.175 

4.351 

70.015 

9 

1.072 

3.971 

73.986 


The table 4 above shows that there are 9 factors of half positive and half negative 
form of items and they contributed 73.98% of cumulative. 

Table 5 Random [Positive & Negative] Form Factor Analysis 


Component 

Total 

% of Variance 

Cumulative % 

1 

5.265 

19.501 

19.501 

2 

3.066 

11.354 

30.855 

3 

2.488 

9.216 

40.071 

4 

2.191 

8.116 

48.187 

5 

1.886 

6.986 

55.172 

6 

1.591 

5.892 

61.064 

7 

1.493 

5.531 

66.595 

8 

1.174 

4.347 

70.942 


The table 4 above shows that there are 8 factors of random positive and negative 
form of items and they contributed 76.32% of cumulative. 

2. Second Objective 

To exa min e the effect of items direction (positive or negative) on the 
criterion related validity in Likert scale. 

Table 6 Criterion Related Validity Coefficient 


Items Direction 

Criterion related Validity Coefficient 

Positive 

0.54 

Negative 

0.68 

Half positive and Half- negative 

0.61 

Random [positive & negative] 

0.66 
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Table no. {6} showed that there are differences between the validity coefficients, 
but we need to use the Fisher equation to know whether this difference is 
statistically significant or not. 

Table 7 Fisher Coefficient 


Comparison 

Calculated (f ) 

F table 

Sig 

Positive * Negative 

1.24 

1.96 

Not 

Positive * Half positive and 
Half negative 

0.26 

1.96 

Not 

Positive * Random [positive & 
negative] 

1.11 

1.96 

Not 

Negative * Half positive and 
Half negative 

0.62 

1.96 

Not 

Negtive * Random [positive & 
negative] 

0.18 

1.96 

Not 

Half positive and Half negative 
* Random [positive & 
negative] 

0.46 

1.96 

Not 


Tablet (7) revealed that fisher values are not significant and the validity 
coefficient calculated by used the Pearson correlation coefficient are not affected 
even when the items’ directions for Likert’s type differ (positive, negative, half- 
positive, half- negative and random [positive & negative]). Thus the hypothesis, 
“there is no effect of items direction (positive or negative) on the criterion related 
validity in Likert scale” was accepted. 


CONCLUSION 

It is thus concluded that analysis revealed that the selected items direction 
(positive or negative) have effect on the factorial construction and criterion 
related validity in Likert scale. The investigator has drawn some conclusions 
which are being presented below:- 

1. It can be concluded that there are difference in the number of factors 
calculated from factor analysis and showed that there is effect of items direction 
(positive or negative) on the factorial construction related validity in Likert scale. 

2. It can be concluded that the criterion related validity is not affected even 
when the items’ directions for Likert’s type differ (positive, negative, half- 
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positive, half- negative and random [positive & negative]) and there is no effect 
of items direction (positive or negative) on the criterion related validity in Likert 
scale. 
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Summary 

Effect of Items Direction (Positive or Negative) on the Factorial 
Construction and Criterion related Validity in Likert Scale 
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The study was conducted to know the effect of items direction (positive or negative) on 
the factorial constmction and criterion related validity in Likert scale. The descriptive 
survey research method was used for the study and the sample consisted of 510 
undergraduate students selected by used random sampling technique. A scale developed 
by SobheiHamdan& Mohammed Jehad (2007) was used by the investigator to access 
attitude of students towards undergraduate study. The finding of the study revealed that: 
(1) there is effect of items direction (positive or negative) on the factorial construction 
related validity in Likert scale; and (2) criterion related validity is not affected even when 
the items’ directions for Likert’s type differ (positive, negative, half- positive, half- 
negative and random [positive & negative]) and there is no effect of items direction 
(positive or negative) on the criterion related validity in Likert scale. 
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